Pandas Profiling Report

Dataset statistics

Number of variables	10
Number of observations	557
Missing cells	3
Missing cells (%)	0.1%
Duplicate rows	0
Duplicate rows (%)	0.0%
Total size in memory	41.4 KiB
Average record size in memory	76.1 B

Variable types

NUM	9
CAT	1

Reproduction

Analysis started	2020-06-03 11:44:21.559886
Analysis finished	2020-06-03 11:48:40.393350
Duration	4 minutes and 18.83 seconds
Version	pandas-profiling v2.8.0
Command line	`pandas_profiling --config_file config.yaml [YOUR_FILE.csv]`
Download configuration	config.yaml

Warnings

`ocean_proximity` has constant value "NEAR BAY"	Constant
`households` is highly correlated with `total_bedrooms`	High correlation
`total_bedrooms` is highly correlated with `households`	High correlation

longitude
Real number (ℝ)

Distinct count	21
Unique (%)	3.8%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	-122.23524236983842
Minimum	-122.34
Maximum	-122.12
Zeros	0
Zeros (%)	0.0%
Memory size	4.4 KiB

Quantile statistics

Minimum	-122.34
5-th percentile	-122.29
Q1	-122.27
median	-122.25
Q3	-122.2
95-th percentile	-122.16
Maximum	-122.12
Range	0.22
Interquartile range (IQR)	0.07

Descriptive statistics

Standard deviation	0.04335092458
Coefficient of variation (CV)	-0.0003546516024
Kurtosis	-0.7682353449
Mean	-122.2352424
Median Absolute Deviation (MAD)	0.03
Skewness	0.4382330124
Sum	-68085.03
Variance	0.001879302662

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
-122.27	62	11.1%
-122.28	60	10.8%
-122.25	53	9.5%
-122.26	49	8.8%
-122.29	36	6.5%
-122.19	35	6.3%
-122.23	35	6.3%
-122.18	32	5.7%
-122.24	32	5.7%
-122.21	29	5.2%
Other values (11)	134	24.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-122.34	1	0.2%
-122.33	1	0.2%
-122.3	18	3.2%
-122.29	36	6.5%
-122.28	60	10.8%

Value	Count	Frequency (%)
-122.12	2	0.4%
-122.13	5	0.9%
-122.14	4	0.7%
-122.15	6	1.1%
-122.16	19	3.4%

latitude
Real number (ℝ_≥0)

Distinct count	18
Unique (%)	3.2%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	37.81089766606823
Minimum	37.73
Maximum	37.9
Zeros	0
Zeros (%)	0.0%
Memory size	4.4 KiB

Quantile statistics

Minimum	37.73
5-th percentile	37.75
Q1	37.77
median	37.81
Q3	37.85
95-th percentile	37.89
Maximum	37.9
Range	0.17
Interquartile range (IQR)	0.08

Descriptive statistics

Standard deviation	0.04414151472
Coefficient of variation (CV)	0.001167428372
Kurtosis	-0.9011517507
Mean	37.81089767
Median Absolute Deviation (MAD)	0.04
Skewness	0.2635547511
Sum	21060.67
Variance	0.001948473322

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
37.77	54	9.7%
37.81	48	8.6%
37.8	48	8.6%
37.79	46	8.3%
37.78	41	7.4%
37.82	35	6.3%
37.85	33	5.9%
37.75	30	5.4%
37.76	29	5.2%
37.83	28	5.0%
Other values (8)	165	29.6%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
37.73	7	1.3%
37.74	20	3.6%
37.75	30	5.4%
37.76	29	5.2%
37.77	54	9.7%

Value	Count	Frequency (%)
37.9	13	2.3%
37.89	22	3.9%
37.88	26	4.7%
37.87	23	4.1%
37.86	26	4.7%

housing_median_age
Real number (ℝ_≥0)

Distinct count	41
Unique (%)	7.4%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	44.81508078994614
Minimum	2.0
Maximum	52.0
Zeros	0
Zeros (%)	0.0%
Memory size	4.4 KiB

Quantile statistics

Minimum	2
5-th percentile	26
Q1	41
median	49
Q3	52
95-th percentile	52
Maximum	52
Range	50
Interquartile range (IQR)	11

Descriptive statistics

Standard deviation	9.211733341
Coefficient of variation (CV)	0.2055498546
Kurtosis	2.23408609
Mean	44.81508079
Median Absolute Deviation (MAD)	3
Skewness	-1.536954554
Sum	24962
Variance	84.85603115

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
52	231	41.5%
50	23	4.1%
43	23	4.1%
46	23	4.1%
49	23	4.1%
41	18	3.2%
44	16	2.9%
42	16	2.9%
45	15	2.7%
39	14	2.5%
Other values (31)	155	27.8%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
2	1	0.2%
10	3	0.5%
13	1	0.2%
14	1	0.2%
15	1	0.2%

Value	Count	Frequency (%)
52	231	41.5%
51	6	1.1%
50	23	4.1%
49	23	4.1%
48	12	2.2%

total_rooms
Real number (ℝ_≥0)

Distinct count	516
Unique (%)	92.6%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	1865.5098743267504
Minimum	12.0
Maximum	12842.0
Zeros	0
Zeros (%)	0.0%
Memory size	4.4 KiB

Quantile statistics

Minimum	12
5-th percentile	503.8
Q1	1087
median	1668
Q3	2324
95-th percentile	3770.8
Maximum	12842
Range	12830
Interquartile range (IQR)	1237

Descriptive statistics

Standard deviation	1156.893331
Coefficient of variation (CV)	0.6201485967
Kurtosis	16.11583934
Mean	1865.509874
Median Absolute Deviation (MAD)	630
Skewness	2.57008092
Sum	1039089
Variance	1338402.178

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
880	2	0.4%
954	2	0.4%
2492	2	0.4%
2252	2	0.4%
2269	2	0.4%
1813	2	0.4%
2239	2	0.4%
909	2	0.4%
1420	2	0.4%
2315	2	0.4%
Other values (506)	537	96.4%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
12	1	0.2%
96	1	0.2%
105	1	0.2%
135	1	0.2%
142	1	0.2%

Value	Count	Frequency (%)
12842	1	0.2%
7355	1	0.2%
7099	1	0.2%
5963	1	0.2%
5871	1	0.2%

total_bedrooms
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct count	379
Unique (%)	68.4%
Missing	3
Missing (%)	0.5%
Infinite	0
Infinite (%)	0.0%

Mean	409.2184115523466
Minimum	4.0
Maximum	2477.0
Zeros	0
Zeros (%)	0.0%
Memory size	4.4 KiB

Quantile statistics

Minimum	4
5-th percentile	124.95
Q1	237
median	364
Q3	480.75
95-th percentile	865.75
Maximum	2477
Range	2473
Interquartile range (IQR)	243.75

Descriptive statistics

Standard deviation	287.2340961
Coefficient of variation (CV)	0.7019090247
Kurtosis	13.78927144
Mean	409.2184116
Median Absolute Deviation (MAD)	125
Skewness	2.934365032
Sum	226707
Variance	82503.42599

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
195	6	1.1%
460	6	1.1%
293	5	0.9%
263	4	0.7%
184	4	0.7%
261	4	0.7%
300	4	0.7%
574	4	0.7%
335	4	0.7%
391	4	0.7%
Other values (369)	509	91.4%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
4	1	0.2%
29	1	0.2%
31	2	0.4%
38	1	0.2%
42	1	0.2%

Value	Count	Frequency (%)
2477	1	0.2%
2408	1	0.2%
2048	1	0.2%
1914	1	0.2%
1750	1	0.2%

population
Real number (ℝ_≥0)

Distinct count	466
Unique (%)	83.7%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	968.6840215439856
Minimum	18.0
Maximum	4985.0
Zeros	0
Zeros (%)	0.0%
Memory size	4.4 KiB

Quantile statistics

Minimum	18
5-th percentile	321
Q1	582
median	863
Q3	1171
95-th percentile	1931.8
Maximum	4985
Range	4967
Interquartile range (IQR)	589

Descriptive statistics

Standard deviation	580.3511967
Coefficient of variation (CV)	0.5991130067
Kurtosis	8.811854949
Mean	968.6840215
Median Absolute Deviation (MAD)	287
Skewness	2.217827901
Sum	539557
Variance	336807.5115

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
849	3	0.5%
496	3	0.5%
1005	3	0.5%
901	3	0.5%
506	3	0.5%
582	3	0.5%
718	3	0.5%
1085	3	0.5%
987	3	0.5%
1481	3	0.5%
Other values (456)	527	94.6%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
18	1	0.2%
86	1	0.2%
94	1	0.2%
98	1	0.2%
125	1	0.2%

Value	Count	Frequency (%)
4985	1	0.2%
4367	1	0.2%
3741	1	0.2%
3668	1	0.2%
3469	1	0.2%

households
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct count	376
Unique (%)	67.5%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	385.8563734290844
Minimum	7.0
Maximum	2323.0
Zeros	0
Zeros (%)	0.0%
Memory size	4.4 KiB

Quantile statistics

Minimum	7
5-th percentile	120.6
Q1	223
median	339
Q3	460
95-th percentile	804.6
Maximum	2323
Range	2316
Interquartile range (IQR)	237

Descriptive statistics

Standard deviation	269.9745009
Coefficient of variation (CV)	0.6996761476
Kurtosis	12.64546671
Mean	385.8563734
Median Absolute Deviation (MAD)	120
Skewness	2.838959545
Sum	214922
Variance	72886.23113

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
259	5	0.9%
355	5	0.9%
244	4	0.7%
335	4	0.7%
422	4	0.7%
155	4	0.7%
350	4	0.7%
340	4	0.7%
133	4	0.7%
255	3	0.5%
Other values (366)	516	92.6%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
7	1	0.2%
23	1	0.2%
34	1	0.2%
39	1	0.2%
47	2	0.4%

Value	Count	Frequency (%)
2323	1	0.2%
2051	1	0.2%
1967	1	0.2%
1789	1	0.2%
1742	1	0.2%

median_income
Real number (ℝ_≥0)

Distinct count	530
Unique (%)	95.2%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	3.382615798922801
Minimum	0.4999
Maximum	13.499
Zeros	0
Zeros (%)	0.0%
Memory size	4.4 KiB

Quantile statistics

Minimum	0.4999
5-th percentile	1.18998
Q1	2.0938
median	2.8321
Q3	4.0469
95-th percentile	7.49056
Maximum	13.499
Range	12.9991
Interquartile range (IQR)	1.9531

Descriptive statistics

Standard deviation	1.977647742
Coefficient of variation (CV)	0.5846504184
Kurtosis	3.452196142
Mean	3.382615799
Median Absolute Deviation (MAD)	0.8983
Skewness	1.634949641
Sum	1884.117
Variance	3.911090592

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
2.875	4	0.7%
3.875	3	0.5%
3.225	2	0.4%
2.6429	2	0.4%
2.5139	2	0.4%
1.9028	2	0.4%
4.7708	2	0.4%
1.1667	2	0.4%
2.537	2	0.4%
2.125	2	0.4%
Other values (520)	534	95.9%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0.4999	1	0.2%
0.7286	1	0.2%
0.75	1	0.2%
0.76	1	0.2%
0.7683	1	0.2%

Value	Count	Frequency (%)
13.499	1	0.2%
12.3804	1	0.2%
12.2138	1	0.2%
11.8603	1	0.2%
11.6017	1	0.2%

median_house_value
Real number (ℝ_≥0)

Distinct count	455
Unique (%)	81.7%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	198326.22980251347
Minimum	60000.0
Maximum	500001.0
Zeros	0
Zeros (%)	0.0%
Memory size	4.4 KiB

Quantile statistics

Minimum	60000
5-th percentile	82080
Q1	112800
median	169300
Q3	259600
95-th percentile	391240
Maximum	500001
Range	440001
Interquartile range (IQR)	146800

Descriptive statistics

Standard deviation	103369.5674
Coefficient of variation (CV)	0.5212097639
Kurtosis	0.235843744
Mean	198326.2298
Median Absolute Deviation (MAD)	66500
Skewness	0.9526447519
Sum	110467710
Variance	1.068526747e+10

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
500001	10	1.8%
137500	7	1.3%
112500	6	1.1%
162500	5	0.9%
125000	5	0.9%
150000	4	0.7%
350000	4	0.7%
98200	3	0.5%
140600	3	0.5%
216700	3	0.5%
Other values (445)	507	91.0%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
60000	1	0.2%
67500	1	0.2%
70000	1	0.2%
71300	1	0.2%
72000	1	0.2%

Value	Count	Frequency (%)
500001	10	1.8%
489600	1	0.2%
483300	1	0.2%
471600	1	0.2%
466100	1	0.2%

ocean_proximity
Categorical

CONSTANT
REJECTED

Distinct count	1
Unique (%)	0.2%
Missing	0
Missing (%)	0.0%
Memory size	2.2 KiB

NEAR BAY	557

Common Values
Length

Value	Count	Frequency (%)
NEAR BAY	557	100.0%

Length

Max length	8
Median length	8
Mean length	8
Min length	8

Pearson's r

The Pearson's correlation coefficient (r) is a measure of linear correlation between two variables. It's value lies between -1 and +1, -1 indicating total negative linear correlation, 0 indicating no linear correlation and 1 indicating total positive linear correlation. Furthermore, r is invariant under separate changes in location and scale of the two variables, implying that for a linear function the angle to the x-axis does not affect r.

To calculate r for two variables X and Y, one divides the covariance of X and Y by the product of their standard deviations.

Spearman's ρ

The Spearman's rank correlation coefficient (ρ) is a measure of monotonic correlation between two variables, and is therefore better in catching nonlinear monotonic correlations than Pearson's r. It's value lies between -1 and +1, -1 indicating total negative monotonic correlation, 0 indicating no monotonic correlation and 1 indicating total positive monotonic correlation.

To calculate ρ for two variables X and Y, one divides the covariance of the rank variables of X and Y by the product of their standard deviations.

Kendall's τ

Similarly to Spearman's rank correlation coefficient, the Kendall rank correlation coefficient (τ) measures ordinal association between two variables. It's value lies between -1 and +1, -1 indicating total negative correlation, 0 indicating no correlation and 1 indicating total positive correlation.

To calculate τ for two variables X and Y, one determines the number of concordant and discordant pairs of observations. τ is given by the number of concordant pairs minus the discordant pairs divided by the total number of pairs.

Phik (φk)

Phik (φk) is a new and practical correlation coefficient that works consistently between categorical, ordinal and interval variables, captures non-linear dependency and reverts to the Pearson correlation coefficient in case of a bivariate normal input distribution. There is extensive documentation available here.

First rows

	longitude	latitude	housing_median_age	total_rooms	total_bedrooms	population	households	median_income	median_house_value	ocean_proximity
0	-122.23	37.88	41.0	880.0	129.0	322.0	126.0	8.3252	452600.0	NEAR BAY
1	-122.22	37.86	21.0	7099.0	1106.0	2401.0	1138.0	8.3014	358500.0	NEAR BAY
2	-122.24	37.85	52.0	1467.0	190.0	496.0	177.0	7.2574	352100.0	NEAR BAY
3	-122.25	37.85	52.0	1274.0	235.0	558.0	219.0	5.6431	341300.0	NEAR BAY
4	-122.25	37.85	52.0	1627.0	280.0	565.0	259.0	3.8462	342200.0	NEAR BAY
5	-122.25	37.85	52.0	919.0	213.0	413.0	193.0	4.0368	269700.0	NEAR BAY
6	-122.25	37.84	52.0	2535.0	489.0	1094.0	514.0	3.6591	299200.0	NEAR BAY
7	-122.25	37.84	52.0	3104.0	687.0	1157.0	647.0	3.1200	241400.0	NEAR BAY
8	-122.26	37.84	42.0	2555.0	665.0	1206.0	595.0	2.0804	226700.0	NEAR BAY
9	-122.25	37.84	52.0	3549.0	707.0	1551.0	714.0	3.6912	261100.0	NEAR BAY

Last rows

	longitude	latitude	housing_median_age	total_rooms	total_bedrooms	population	households	median_income	median_house_value	ocean_proximity
547	-122.27	37.77	52.0	1710.0	481.0	849.0	457.0	2.7115	220800.0	NEAR BAY
548	-122.27	37.77	52.0	2388.0	559.0	1121.0	518.0	3.3269	234500.0	NEAR BAY
549	-122.25	37.77	52.0	2156.0	458.0	872.0	445.0	3.2685	254200.0	NEAR BAY
550	-122.26	37.77	52.0	1565.0	315.0	637.0	297.0	4.7778	351800.0	NEAR BAY
551	-122.26	37.77	52.0	1704.0	371.0	663.0	340.0	4.2260	275000.0	NEAR BAY
552	-122.26	37.77	52.0	1210.0	168.0	411.0	172.0	3.3571	405400.0	NEAR BAY
553	-122.26	37.77	52.0	2097.0	444.0	915.0	413.0	2.9899	228100.0	NEAR BAY
554	-122.26	37.77	52.0	1848.0	479.0	921.0	477.0	2.8750	234000.0	NEAR BAY
555	-122.24	37.77	43.0	955.0	284.0	585.0	266.0	2.3882	162500.0	NEAR BAY
556	-122.25	37.77	43.0	4329.0	1110.0	2086.0	1053.0	2.9750	243400.0	NEAR BAY

Overview

Variables

Interactions

Correlations

Pearson's r

Spearman's ρ

Kendall's τ

Phik (φk)

Missing values

Sample

First rows

Last rows